The present invention relates to animation production. More specifically, the present invention relates to systems and methods for making and storing visual annotations to a video signal for directorial review.
Throughout the years, movie makers have often tried to tell stories involving make-believe creatures, far away places, and fantastic things. To do so, they have often relied on animation techniques to bring the make-believe to “life.” Two of the major paths in animation have traditionally included, drawing-based animation techniques and stop motion animation techniques.
Drawing-based animation techniques were refined in the twentieth century, by movie makers such as Walt Disney and used in movies such as “Snow White and the Seven Dwarfs” (1937) and “Fantasia” (1940). This animation technique typically required artists to hand-draw (or paint) animated images onto a transparent media or cels. After painting, each cel would then be captured or recorded onto film as one or more frames in a movie.
Stop motion-based animation techniques typically required the construction of miniature sets, props, and characters. The filmmakers would construct the sets, add props, and position the miniature characters in a pose. After the animator was happy with how everything was arranged, one or more frames of film would be taken of that specific arrangement. Stop motion animation techniques were developed by movie makers such as Willis O'Brien for movies such as “King Kong” (1933). Subsequently, these techniques were refined by animators such as Ray Harryhausen for movies including “Mighty Joe Young” (1948) and Clash Of The Titans (1981).
With the wide-spread availability of computers in the later part of the twentieth century, animators began to rely upon computers to assist in the animation process. This included using computers to facilitate drawing-based animation, for example, by painting images, by generating in-between images (“tweening”), and the like. This also included using computers to augment stop motion animation techniques. For example, physical models could be represented by virtual models in computer memory, and manipulated.
One of the pioneering companies in the computer aided animation (CAA) industry was Pixar, dba Pixar Animation Studios. Over the years, Pixar developed and offered both computing platforms specially designed for CAA, and Academy-Award® winning rendering software known as RenderMan®.
In the filmmaking process, a Director must constantly review the visual data being created and provide feedback. This criticism can be conveyed in many ways. Different types of directorial suggestions may use different means of expression. For example, a suggestion about the “mood” of a character may be described by physical acting or by verbal description. However, many filmic elements are visual in nature, and rely on visual feedback. For example, if a Director would like to move a character to a specific location on screen, that suggestion is often conveyed more clearly and efficiently by “showing” the location in the scene. The inventors of the present invention have discovered that advantages of visual feedback are even more important in the domain of computer graphics and animation. In such domains, the Director must convey extremely specific visual information about a character's physical appearance.
The inventors of the present invention had determined several desired traits for visual annotation tools. First, the inventors have determined that methods and systems should be simple and intuitive, because Directors are extremely busy and do not have time to learn how to use computers or learn how to use complicated software. Second, the inventors have determined that the annotation methods and tools should be fast, lightweight, and fit into the current review workflow. Typically, review sessions are fast-paced and cover much material, accordingly, the annotation logistics should be simple. Third, the inventors have determined that annotation tools should be compatible with different types of visual signals. For example, in computer animation, work to be reviewed is sometimes reviewed from digital video, the artist's computer screen, or other source. Finally, the inventors have determined that visual annotation tools should provide users with easy access to the Director's critiques so that users can make appropriate changes and refinements.
At Pixar, two methods were previously used to give users (e.g. animators) visual feedback. One method was for an image to be projected on a screen, and for the Director to use a laser pointer to visually indicate the problem areas.
Drawbacks to this solution was that it was only effective for simple changes and edits, such as, rough positioning of objects in a scene. This solution could not be used to describe complex shapes or changes. Additionally, this solution did not allow the Director's comments to be saved for subsequent review.
Another method was for an image to be projected onto a whiteboard and for the Director to make sketches on the whiteboard. Once the sketch was complete, a digital photograph was taken for later reference.
Drawbacks to this method included that there was significant overhead and it greatly impacted the Director review workflow. For example, reviews were limited to small audiences and small rooms due to the limitations of the whiteboard size. Additionally, projecting images onto a glossy whiteboard significantly diminished the quality of the images. Other drawbacks included that the comments written on the whiteboard could not easily be correlated to the image that was projected. More specifically, a substantial amount of time would have to be spent photographing the images on the white board, organizing the photographs, correlating the images to the projected images, and distributing the projected images and the photographs to users.
In the industry, a number of computerized techniques for annotating information streams currently exists. One approach is to have the user draft textual or graphic notes while viewing a video clip, and record a time stamp in the video clip the moment the note was created. Subsequently, when the notes are reviewed, using the time stamp, the appropriate place in the video clip is located.
These techniques include many significant drawbacks. One drawback is that it imposes a strictly linear notation strategy on the review process. In other words, the Director would have to annotate the video as it were being shown. If the Director made comments regarding a portion of the video and later wanted to make further comments on the portion of the video, a new time stamp would be recorded for the portion of the video. To a user making revisions, changes to the portion would be made twice: once in response to the Director's first comments, and once, later, in response to the further comments.
Another drawback is that these techniques require knowledge of the underlying file format for the video clip in order to insert the time stamps. In other words, the sources of the video data are limited to ones which are output by a computer. Further, to support additional and new computerized video formats would require significant reengineering. Accordingly, these techniques are very limiting and inflexible.
Accordingly, what is desired are improved methods and apparatus addressing the drawbacks discussed above.